Efficient two-sample functional estimation and the super-oracle phenomenon

We consider the estimation of two-sample integral functionals, of the type that occur naturally, for example, when the object of interest is a divergence between unknown probability densities. Our first main result is that, in wide generality, a weighted nearest neighbour estimator is efficient, in the sense of achieving the local asymptotic minimax lower bound. Moreover, we also prove a corresponding central limit theorem, which facilitates the construction of asymptotically valid confidence intervals for the functional, having asymptotically minimal width. One interesting consequence of our results is the discovery that, for certain functionals, the worst-case performance of our estimator may improve on that of the natural `oracle' estimator, which is given access to the values of the unknown densities at the observations.Comment: 82 page

A useful variant of the Davis--Kahan theorem for statisticians

The Davis--Kahan theorem is used in the analysis of many statistical procedures to bound the distance between subspaces spanned by population eigenvectors and their sample versions. It relies on an eigenvalue separation condition between certain relevant population and sample eigenvalues. We present a variant of this result that depends only on a population eigenvalue separation condition, making it more natural and convenient for direct application in statistical contexts, and improving the bounds in some cases. We also provide an extension to situations where the matrices under study may be asymmetric or even non-square, and where interest is in the distance between subspaces spanned by corresponding singular vectors.Comment: 12 page

Importance Tempering

Simulated tempering (ST) is an established Markov chain Monte Carlo (MCMC) method for sampling from a multimodal density $\pi(\theta)$. Typically, ST involves introducing an auxiliary variable $k$ taking values in a finite subset of $[0,1]$ and indexing a set of tempered distributions, say $\pi_k(\theta) \propto \pi(\theta)^k$. In this case, small values of $k$ encourage better mixing, but samples from $\pi$ are only obtained when the joint chain for $(\theta,k)$ reaches $k=1$. However, the entire chain can be used to estimate expectations under $\pi$ of functions of interest, provided that importance sampling (IS) weights are calculated. Unfortunately this method, which we call importance tempering (IT), can disappoint. This is partly because the most immediately obvious implementation is na\"ive and can lead to high variance estimators. We derive a new optimal method for combining multiple IS estimators and prove that the resulting estimator has a highly desirable property related to the notion of effective sample size. We briefly report on the success of the optimal combination in two modelling scenarios requiring reversible-jump MCMC, where the na\"ive approach fails.Comment: 16 pages, 2 tables, significantly shortened from version 4 in response to referee comments, to appear in Statistics and Computin

EDITORIAL: MEMORIAL ISSUE FOR CHARLES STEIN

The Institute of Mathematical Statistics (IMS) Council approved a proposal from its Committee on Memorials to dedicate this issue of the Annals of Statistics to Charles M. Stein, who died in 2016 aged 96. This memorialisation is a reflection of Stein’s distinction as a mathematical statistician, whose work continues to have a profound impact on the discipline

Editorial: Special Issue on "Nonparametric Inference Under Shape Constraints"

Shape-constrained inference usually refers to nonparametric function estimation and uncertainty quantification under qualitative shape restrictions such as monotonicity, convexity, log-concavity and so on. One of the earliest contributions to the field was by Grenander (1956). Motivated by the theory of mortality measurement, he studied the nonparametric maximum likelihood estimator of a decreasing density function on the nonnegative half-line. A great attraction of this estimator is that, unlike other nonparametric density estimators such as histograms or kernel density estimators, there are no tuning parameters (e.g., bandwidths) to choose.R. J. Samworth is supported by EPSRC Grants EP/P031447/1 and EP/N031938/1. B. Sen is supported by NSF Grants DMS-17-12822 and AST-16-14743